Gestural touch interface

ABSTRACT

A touch-based gesture interface is provided. An initial point of contact for a touch made on a touch-screen surface is associated with a reference value. If the initial contact is sustained before release, directional axis changes for the touch are noted until the contact is released to identify a gesture-based touch input. The directional axis changes associated with the gesture-based touch input are translated into a key entry based on the directional axis changes made relative to the reference value. If the initial contact is released with no directional axis movement/changes, the key entry is assigned to the reference value for the gesture-based touch input. In an embodiment, two successive touches with no directional axis movement/changes made within a threshold period of time is identified as a cancel key.

BACKGROUND

Increasingly traditional keypads are being removed from electronicdevices and terminals. Input from users is being driven by touch-screendisplays, which have been miniaturized and are integrated into even thesmallest of devices, such as smart watches. Elimination ofkeypads/keyboards removes the number of electromechanical and softwarecomponents needed for producing and maintaining the devices/terminals.

However, Automated Teller Machines (ATMs) are one type of device wherePersonal Identification Number (PIN) keypads still have a substantialpresence in the industry; this is for a variety of reasons.

First, ATMs are secure devices that allow consumers to withdraw currencywhile performing withdrawal transactions. Software that processes on theATMs is often the target of thieves for obvious reasons. Thus, it iscrucial that the ATM maintain a secure mechanism by which a user cansecurely enter a PIN for access to a user's account. Typically, an ATMPIN pad is referred to as an encrypted PIN pad that includes its ownindependent processor from the ATM's core processor. When a user entersa PIN, the ATM core processor cannot detect which keys are depressed fora PIN entry; rather, the processor of the encrypted PIN pad receives thePIN and encrypts the PIN with a transaction-specific hash. The encryptedPIN is forwarded from the processor of the encrypted PIN pad to the ATMcore processor and the encrypted PIN is sent from the ATM to a servicingfinancial institution that verifies the encrypted PIN and provides backa validation to the ATM core processor. As a result, the encrypted PINpad provides a secure mechanism by which PINs are received and processedby an ATM.

Second, ATMs are subject to a variety of governmental/agency laws andregulations. One important regulation in the United States is theAmerican's Disability Act (ADA), which mandates that the ATM provide amechanism by which sight-and-hearing-impaired individuals can conducttransactions on the ATM. The encrypted PIN pad typically includes raiseddots on the surfaces of the keys, which can be felt by a sight-impairedindividual for purposes of entering a user PIN. The ATMs also include aheadphone jack for use with headphones to provide audio feedback to thesight-impaired individual during a transaction.

Thus, removal of the encrypted PIN pads has remained an elusive goalthat faces many technological hurdles (security based and compliancebased).

SUMMARY

In various embodiments, methods and a terminal for gestural touchinterface are presented.

According to an embodiment, a method for processing a gestural touchinterface on a touch display is presented. Specifically, and in oneembodiment, a touch made on a touchscreen is associated with a referencevalue. Any changes made in directions to the touch are tracked untilcontact with a surface of the touchscreen is released and agesture-based touch is recognized. A key entry value is assigned to thegesture-based touch based on the changes made in the directions, if any,relative to the reference value.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a diagram of a system for processing a gestural touchinterface on a touch display, according to an example embodiment.

FIG. 1B is an example set of windows illustrating numerical touch-basedgestures for a gestural touch interface, according to an exampleembodiment.

FIG. 2 is a diagram of a method for processing a gestural touchinterface on a touch display, according to an example embodiment.

FIG. 3 is a diagram of another method for processing a gestural touchinterface on a touch display, according to an example embodiment.

FIG. 4 is a diagram of a transaction terminal for processing a gesturaltouch interface on a touch display, according to an example embodiment.

DETAILED DESCRIPTION

FIG. 1A is a diagram of a system 100 for processing a gestural touchinterface on a touch display, according to an example embodiment. It isto be noted that the components are shown schematically in greatlysimplified form, with only those components relevant to understanding ofthe embodiments being illustrated.

Furthermore, the various components (that are identified in the FIG. 1A)are illustrated and the arrangement of the components is presented forpurposes of illustration only. It is to be noted that other arrangementswith more or fewer components are possible without departing from theteachings of processing a gestural touch interface on a touch displaypresented herein and below.

As will be more completely discussed herein and below, the teachingsprovide a gestural (gesture-based) touch interface rendered on atouchscreen that permits touch-gestures to be made on a surface of thetouchscreen for purposes of providing input keys to another service(such as a transaction service, an authentication service, etc.). Thelocation on the surface of the touchscreen that is touched at the startof each gesture-based touch is irrelevant, each such initial touch isassumed to begin at a reference key location associated with a referencekey. Each gesture-based touch is identified when the user releasescontact from the surface of the touchscreen (e.g., lifts finger off thesurface to break contact with the surface), and each gesture-based touchis determined based on the number and types of movements made in avertical and/or horizontal direction. If there are no movements made inany direction from the point of initial contact until release (lift) ofthe finger, then this is considered to be associated with the referencekey. This allows sight-impaired individuals to more easily operatedevices using a mental mapping of a known keypad layout or knownselection menu layout that includes the reference key and its positionrelative to other reference keys in the keypad or known selection menu.The sight-impaired individual requires little to no training to fullyinteract with the gesture-based touch interface because they are alreadyfamiliar with the known keypad layout or the known selection menu.

The system 100 includes a transaction terminal 110. Transaction terminal110 includes: a touchscreen display 120, a transaction processor 130 anda secure encrypted PIN processor 140.

Transaction terminal 110 also includes a non-transitorycomputer-readable storage medium/memory 111 including executableinstructions representing a transaction manager 131. Processor 130execute the executable instructions from the non-transitorycomputer-readable storage medium 111 while processing transactionsperformed on transaction terminal 110.

Furthermore, transaction terminal 110 includes a secure non-transitorycomputer-readable storage medium/memory 140 accessible to the secureprocessor 140. The secure medium/memory 140 including executableinstructions representing a gestural PIN translator 141 and forperforming cryptographic operations (including encryption). Secureprocessor 140 executes the gestural PIN translator 141 and thecryptographic operations from the secure medium/memory 140.

During a transaction being processed on terminal 110, the transactionmanager 131 raises an event indicating that a secure PIN is required tobe entered to continue with the transaction on terminal 110. This causesprocessor 130 to send a message to secure processor 140 requesting a PINvalue. Secure processor 140 then takes processing control over the touchscreen display 120, this prevents the transaction processor 130 frombeing able to access or read any display buffers associated with thetouch-screen display 120. In this way, a user can enter a PIN using anencrypted and secure mechanism on the same touch screen display 120 thatwas being used to conduct the transaction and any software that may havebeen corrupted on terminal 110 is unable to decipher the actual PINentry that is being entered through touch screen display 120 whilesecure processor 140 has operating control over display 120.

The Gestural PIN translator 141 is initiated for processing by secureprocessor 140 when secure processor 140 detects that a headphone cord isinserted into a headphone/speaker jack/port at the terminal 110. Notethat this check may not be needed by secure processor 140 if transactionprocessor 130 indicates with the event that a PIN is being requestedwhere the headphone is already inserted into the jack. The presence ofthe headphone connection indicates that a sight-impaired individual isperforming a transaction at terminal 110. This also triggers audiofeedback processing supplied as audio through the headphones by bothtransaction processor 130 and secure processor 140 (when secureprocessor has exclusive control for PIN entry at terminal 110).

If there is no indication of any connected headphone at terminal 110,then gestural pin translator 141 (or a different software moduleassociated with the processing environment of secure processor 140)renders a numerical keypad on the touchscreen 120. Touch data/eventsthat correspond to numbers on the rendered numerical keypad aretranslated into numbers. Typically, the numbers are custom encryptedwith a transaction identifier hash and provided as an encrypted PINvalue back to the transaction manager 131 of transaction processor 130.Secure processor 140 then relinquishes operating/processing control oftouchscreen 120 back to transaction processor 130 to continuetransaction processing with the encrypted PIN value.

Gestural PIN translator 141, when activated by secure processor 140 andwhen a connected headphone is detected, waits to receive touchdata/events from touchscreen 120 as provided by the user for purposes oftranslating touches provided by the user into number entries for auser's PIN.

FIG. 1B is an example set of windows illustrating numerical touch-basedgestures for a gestural touch interface, according to an exampleembodiment.

Touch data/events are translated by gestural PIN translator 141 inaccordance with the numerical values and corresponding touch movementsillustrated in the FIG. 1B. The touch gestures map to a keypad layoutsuch that memorization and complex touch movements are not needed by thesight-impaired individual.

For example, a center of a keypad is a key for the number 5, thecorresponding touch recognized by gestural PIN translator 141 for thenumber 5 is a single pressed point by a user's finger on the surface oftouchscreen 120 that is then released (lifted) by the user.

It is to be noted that the location of the single pressed point does nothave to be centered within touchscreen 120 and can occur anywhere (suchas top left, bottom right, top right, bottom left, off centered, etc.).

It is also to be noted that although the FIG. 1B illustrates a numerickeypad, touchscreen 120 includes no such rendering, in fact, when theuser provides the PIN, the touchscreen 120 maybe completely blank withnothing rendered visibly on the touchscreen at all. In an embodiment, itis irrelevant as to whether the touchscreen 120 is rendering any visualinformation when gestural PIN translator 141 is activated and awaiting aPIN entry; any currently visual information of touchscreen 120 isdisregarded by gestural PIN translator 141 as gestural PIN translator141 waits for entry of gesture-based touches from the user.

According to this particular mapping, which resembles a common keyboardlayout, the number 5 serves is chosen to serve as a mental referencepoint for the user, all other numbers (except the number 0) can beachieved with the touch associated with a 5. The keypad can bevisualized by the user by simply knowing that a number 5 is thereference location on a virtual keypad.

Accordingly, using the number 5 as a reference point, a lexicon ofgestures can be inferred such that the user can enter a number 1 bymaking a gesture on the surface of touchscreen 120 as one continuoustouch that presses on an initial point (again this initial point can beoffset anywhere on the touchscreen surface), maintaining a continuoustouch (continuous contact) with the surface of touchscreen 120 andmoving to the left and then up or moving up and then to the left, andreleasing (lifting finger off the surface) the continuous touch. With 5as a reference point, there are two paths to reach the 1 key: to theleft and up or up and to the left. The number 2 is entered with agesture-based touch that is pressed on any point of the surface oftouchscreen 120, moves up, and is released. The number 3 is entered witha gesture-based touch that is pressed, moved to the right, moved up, andreleased; or pressed, moved up, moved to the right, and released. Thenumber 4 is entered with a gesture-based touch that is pressed, moved tothe left, and released. The number 5 is entered with a gesture-basedtouch that is pressed, moved to the right, and released. The number 7 isentered with a gesture-based touch that is pressed, moved to the left,moved down, and released; or pressed, moved down, moved to the left, andreleased. The number 8 is entered with a gesture-based touch that ispressed, moved down, and released. The number 9 is entered with agesture-based touch that is pressed, moved down, moved to the right, andreleased; or pressed, moved to the right, moved down, and released.

To the extent that their gestural combination cannot be inferred easilyfrom their location, three keys in the particular configuration of FIG.1B are special keys that are defined by gestural PIN translator 141using a change of direction gesture-based touch movement (represented bythe squares in the FIG. 1B). As shown, the number 0 (zero) is agesture-based touch that is pressed, moved down, then moved up andreleased (lifted). The enter key is a gesture-based touch that can beentered in one of two manners: a press of the finger on any point of thetouchscreen 120 that is then moved to the left, moved to the right, andreleased (lifted); or a press of the finger on any point of touchscreen120 that is then moved to the right, moved to the left, and released. Acancel key is a gesture-based touch that is a double tap on any point oftouchscreen 120. In order to ensure that the cancel key entry is notmistaken for two separate entries for the 5 key, a timer can be setfollowing detection of the first tap and when the second tap is receivedwithin a threshold time (very short period of time), the cancel keyentry is assumed as opposed to identifying two entries of a 5 key.

In an embodiment, two cancel keys entered in succession (2 double tapsback to back) causes the transaction terminal 110 or SST 110 to cancelany user-ongoing transaction and returns any bank card that may beinserted into a card reader of the SST 110.

As the user makes the gesture-based touches for entering a number value(1-9) or key value (0, enter, cancel), the gestural PIN translator 141provides audio based feedback to the user through the connectedheadphone. For example, a short beep, a long beep, beeps with patterns,different toned beeps, different pitched beeps, etc.

In an embodiment, the audio-based feedback provides instructions to theuser when it is detected that a user is using the gesture-based touchprocessing for a first time through the connected headphones. The speechmessages inform the user that the instructions will loop until the userdouble taps anywhere on the touchscreen 120. When the user double taps,the speech-based feedback tells the user it is ready to receive PINinput on the touchscreen 120. In an embodiment, instructions playedthrough headphones includes the following:

“Image the layout of a telephone number pad where the numbers 1, 2, and3 are on a top row; 4, 5, and 6 are on a middle row; 7, 8, and 9 are ona bottom row; and 0 is below 8. To use the gesture-based PIN input, youneed to understand the difference between finding a number and enteringa number. Finding a number is moving your finger to it but not enteringit. After you find a number, it will be entered if you lift your fingeroff the screen. You will hear a low tone when you first touch the screenand a high tone when you enter a number. You will not however hear anysounds as you move your finger around the screen. When you first touchthe screen, your finger will be on the 5 button. It does not matterwhere on the screen you touch: the system will help you by putting the 5button wherever you first touch. Think of this as finding the 5. Toenter a 5, first find it by touching anywhere on the screen, then enterit by lifting your finger off the screen without moving it around. Toenter a different number, you need to find it by moving your finger toits position from the 5. Always start by finding the 5 and then, whilekeeping your finger on the screen move to the number you want beforelifting your finger off the screen to enter it. For example, to enter 1you must first touch the screen to find the 5. Do not lift your fingeryet. Keeping your finger on the screen, move it up in a straight line towhere the 2 would be. You have now found the 2. Now move your fingerleft to where the 1 would be. You have now found the 1. To enter, liftyour finger off the screen now. To enter a number 2, touch the screen tofind the 5. Keep your finger on the glass. Move your finger up to wherethe 2 is. You have found the 2, so lift your finger off the screen toenter it. To enter a 3, touch the screen to find the 5. Keep your fingeron the glass. Move your finger up to where the 2 is to find the 2. Moveyour finger to the right to find the 3. Now lift your finger to enter a3. Enter other numbers using the same method. In each case move yourfinger to the position where the number would be found on a telephonenumber pad. 0 requires a special movement. To enter a 0, first touch thescreen to focus the 5. Move your finger down. Keeping your finger on theglass move it up again back to the 5. Now lift your finger and you willenter a 0. If you make a mistake you can clear and start again by doubletapping anywhere on the screen. To enter a completed PIN number, touchthe screen and move your finger to the right. Without lifting, move yourfinger back to the left. Now lift your finger to enter the completedPIN. It does not matter how big your finger movements are but you needto keep your finger pressed to the glass while you make them and youneed to make horizontal and vertical movements only. Diagonal movementsare not used. These instructions are now complete and will play again.”

In an embodiment, only tones or beeps and non-speech-based feedback isprovided over the headphones when the user is entering PIN key valuesusing the touch-based gestures. This is done for security and to preventany eavesdroppers from hearing PIN key values. This is similar todisplaying asterisks (*'s) in a touchscreen when a user enters apassword or PIN, the asterisks provide a visual feedback that somethingwas entered but does not display what was actually entered. In this way,the PIN is not exposed through any speech-based feedback upon entry bythe user.

Once the user has completed all gesture-based touches for PIN entry andprovided the gesture-based touch for entry, the secure processor 140receives a numerical value for the PIN entry, performs encryption on thenumerical value using a transaction specific hash, and provides theencrypted PIN back to the transaction processor 130 where it is providedto the transaction manager 131 for validation with an appropriatefinancial institution and continued transaction processing.

The gesture-based touches represent gestures that would map to a keypad(if a keypad were available). Thus, a user does not have to memorize thegesture-based touches except for the three special keys (0, enter, andcancel) because a logical mental map of a keyboard is all the user needsand then the user simply makes gestures that correspond to a path forreaching a given number (1-9) using the user's initial point of contacton the touchscreen surface as a point representing the number 5 key.This provides an intuitive interface that would only entail memorizingthree special keys (0, enter, and cancel). The memorization is minimalbecause once the user knows a change of direction (vertical y-axis for 0(zero) and horizontal (horizontal (x-axis)) or double tapped touch(cancel key) is needed with the gesture-based touch for these threespecial keys, the user simply changes direction and make a path eitherfrom that key to the 5 key or from the 5 key to that key. The 0 (zero)key and the enter key are still logically laid out with the specializedgesture-based touches to map to where these keys would be on a keypad,if such keypad were rendered on touchscreen 120; such that only a doubletap for the cancel key requires user memorization.

The gestural PIN translator 141 recognizes the initial point of contactfor a gesture-based touch as representing the 5 key and the uservisualizes the 5 key as the initial touch, if the user does not want toenter a 5 key, the user makes a continuous touch drawing a path from the5 key to the key the user wants to enter and then releases the touch.This maps to how users visualize a keypad. The scale and length of thecontinuous touch movements do not alter the accuracy of the gestural PINtranslator 141 in determining an entered key; rather, the gestural PINtranslator 141 assumes on first contact the user is at a 5 key, suchthat a short move to the left with a long move upward is still the 1 key(or vice versa). The user can determine the scale and length desired,the gestural PIN translator 141 looks for a continuous touch and thedirectional changes noted when the continuous touch is released.Gestural PIN translator 141 is only recognizing the directions ofcontinuous touch encountered once released by the user. Four types ofdirectional movements are recognized: left, right, up, and down. A givensingle gesture-based touch has at most two directional movements alongan x-axis and/or y-axis. Each of the three specialized keys have twodirectional movements along a same axis (x-axis for enter key and cancelkey, and y-axis for zero (0) key). There are only two keys that entailno directional movements, the reference 5 key and the double tappedtouches for the cancel key.

Because the user determines the size of the gestural-based touches,users concerned with onlookers can make very small gestural-basedtouches without effecting the accuracy of gestural PIN translator 141 inrecognizing the PIN key entries supplied by the user.

Additionally, users with some sight often have security concerns aboutexcessively sized keypads rendered on screens for PIN entry. Such userscan use the gesture-based PIN entry approaches here without any securityconcerns and without having to produce a large font rendering of anykeypad on a display.

In an embodiment, a “gesture-based touch” refers to a single touch thatmaintains contact with a surface of a touchscreen 120 and is releasedafter no directional change in touch movement is detected (e.g., the key5) or a double tap of two touches (e.g., cancel key); or is releasedafter one or two directional changes in touch movement is detected. Anytwo directional changes can be any size/length of movement for the touchalong a same axis (e.g., 0 (zero) key or enter key) or a different axis(for both x-axis and y-axis, e.g., 1, 3, 7, and 9 keys). Singledirectional changes (movement in a single direction from the initialpoint of contact) include the 2, 4, 6, and 8 keys. A single touch andrelease includes no directional change and is recognized as a 5 key andis the point of reference for keys 1-4 and 6-9. Two successive touch andreleases is recognized as a cancel key. The magnitude (length) of anytouch movement made in any direction is irrelevant; rather, the changesin direction (if any) are considered when resolving the gestural inputs.

In an embodiment, an initial rendered keypad is provided by secureprocessor 140 when secure processor 140 takes over control for encryptedPIN entry from transaction processor 130. A single double touch anywhereon touchscreen 120 removes the rendering of the keypad and activatesgestural PIN translator 141 for the gesture-based encrypted PIN entry.

In an embodiment, a whited out screen is presented on touchscreen 120when gestural PIN translator 141 is activated for encryptedgesture-based PIN entry and remains until either entry is canceled or aPIN is received, at which time transaction processor 130 regainsprocessing control of touchscreen 120 for completing the transactionprocessing.

In an embodiment, transaction processor 130 may execute a secondinstance of gestural PIN translator 141 within the processing space oftransaction manager 131. This is done independent of the gestural PINtranslator 141 executed by the secure processor 140. So, the entiretransaction from start to finish can be performed by the user throughthe aforementioned gesture-based touches. For example, a sight impaireduser approaches an ATM (a type of transaction terminal 110) and insertsthe user's card and inserts a headphone cord into a headphone jack/portof the ATM. The transaction manager 110 then communicates with the userthrough speech, asking the user to select a type of transaction as 1 fordeposit, 2 for withdrawal, or 3 for account balance. The user doubletaps anywhere on the surface of touchscreen 120 and the second instanceof the gestural PIN translator 141 is activated within the transactionprocessing space controlled by transaction processor 130. Any visualdata previously rendered on the touchscreen 120 is removed. The usermakes a gesture-based touch for selection number 2, the second instanceof the gestural PIN translator 141 communicates the 2 selection totransaction manager 131. The audio feedback then requests that the userselect an account for the withdrawal by communicating through speech,select 1 for checking or 2 for savings. The user makes a gesture-basedtouch on the touchscreen surface for the number 1, the second instanceof the gestural PIN translator 141 communicates to the transactionmanager 131. Transaction manager 131 raises an event to activate secureprocessor 140, touchscreen 120 control is passed from transactionprocessor 130 to secure processor 140. The secure instance of gesturalPIN translator 141 is activated, the user enters the gesture-basedtouches, and secure instance of the gestural PIN translator 141 providesthe numerical value back to secure processor 140. Secure processor 140performs transaction-specific encryption and provides an encrypted PINvalue back to secure processor 130 and then relinquishes control oftouchscreen 120 back to transaction processor 130. Transaction manager131 provides the encrypted PIN to the financial institution over anetwork switch along with the transaction details (such as withdrawal of$100 from account X with encrypted PIN value Y). Assuming the financialinstitution approves of the transaction, an encrypted nonce is sent backto transaction manager 131 over the network switch from the financialinstitution. Transaction manager 131 provides the encrypted nonce to acash dispenser, the cash dispenser validates the encrypted nonce,activates the safe, and dispenses the cash. Audio feedback tells theuser to remove the cash. Speech can be used to see if the user wants tocontinue with another transaction or is completed, if the user wants tocontinue the second instance of the gestural PIN translator 141 is usedfor whatever input is needed form the user for another transaction.

In an embodiment, the transaction terminal 110 is one of: an ATM, aSelf-Service Terminal (SST), and a Point-Of-Sale (POS) terminal.

In an embodiment, gestural PIN translator 141 can be executed on adevice other than a transaction terminal 110 for other purposes ofreceiving input from a sight-impaired individual. For example, thedevice can be a tablet/phone where a numeric input is required from theuser, such as a PIN or passcode (to unlock the device), a currencyamount, etc. The tablet/phone can be operated by a clerk interactingwith the user or be a tablet/phoned operated by the sight-impairedindividual.

In an embodiment, secure processor 140 and gestural PIN translator 141are processed on a transaction terminal 110 that lacks an encrypted PINpad.

In an embodiment, secure processor 140 and gestural PIN translator 141are processed on a transaction terminal 110 that includes an encryptedPIN pad but gestural PIN translator 141 replaces functionality of theencrypted PIN pad when gestural PIN translator 141 is activated. Thatis, the encrypted PIN pad is deactivated by secure processor 140 whensecure processor 140 receives an event that causes secure processor toactivate gestural PIN translator 141.

In an embodiment, touchscreen display 120 does not include and lacks anytactile-based screen overlay or haptic (vibrational or air-based)feedback capabilities.

In an embodiment, any visual model can be used with a starting referencepoint to track the paths from the reference point to a given characteror selection. That is, the discussion above utilized a keypad layoutmodel with 5 being a reference starting point and the gesture-basedtouches being paths from the initial reference point of 5 based on 5'slocation within keypad layout model relative to other keys. So, anyspecialized adding machine/keyboard layouts or even interface selectionitems can be used with a known reference starting location and eachtouch-based gesture representing a path from the reference startinglocation to the desired key/item or selection within the visual model.

In an embodiment, insertion of headphone connection into an audiopost/jack of terminal 110 activates gestural PIN translator 141 andspeech-driven audio feedback. In an embodiment, gestural PIN translator141 is activated after a user is instructed following insertion of theheadphone cord to perform a double tap anywhere on touchscreen 120. Inan embodiment, a specialized button or key with tactile features on anexisting keypad of the terminal 110 can be pressed to activate gesturalPIN translator 141.

These and other embodiments, are now discussed with reference to theFIGS. 2-4.

FIG. 2 is a diagram of a method 200 for processing a gestural touchinterface on a touch display, according to an example embodiment. Thesoftware module(s) that implements the method 200 is referred to as a“gesture-touch screen manager.” The gesture-touch screen manager isimplemented as executable instructions programmed and residing withinmemory and/or a non-transitory computer-readable (processor-readable)storage medium and executed by one or more processors of a device. Theprocessor(s) of the device that executes the gesture-touch screenmanager are specifically configured and programmed to process thegesture-touch screen manager. The gesture-touch screen manager may haveaccess to one or more network connections during its processing. Thenetwork connections can be wired, wireless, or a combination of wiredand wireless.

In an embodiment, the device that executes the gesture-touch screenmanager is transaction terminal 110. In an embodiment, the transactionterminal 110 is one of: a SST, a POS terminal, an ATM, and/or a kiosk:

In an embodiment, the device that executes the gesture-touch screenmanager is one of: a tablet, a phone, a laptop, a desktop, and awearable processing.

In an embodiment, the gesture-touch screen manager is the gestural PINtranslator 141.

At 210, the gesture-touch screen manager detects a touch on atouchscreen of a device. The touch comprises contact with thetouchscreen followed by a release of the contact from the touchscreen.

At 220, the gesture-touch screen manager associates the contact with areference key. That is as soon as a finger touches a surface of thetouchscreen, the gesture-touch screen manager recognizes the touch asthe reference key (e.g., in the above example of the FIGS. 1A-1B, thereference value is the 5 key on a logical/virtual keypad).

In an embodiment, at 221, the gesture-touch screen manager ignoressubsequent touches until an audio device is detected as being connectedto the device. The connection to the audio device can be made bydetecting a headphone inserted into a headphone port of the device thatprocesses the gesture-touch screen manager. In another case, theconnection to the audio device can be made by detecting a wirelessconnection made wireless connected device that changes the audio outputof the device to the wireless connected device. In an embodiment, thegesture-touch screen manager continues to ignore the other touches afterdetection of a connected audio device until the user performs a doubletapped touch (two touches performed within a short threshold period oftime).

In an embodiment of 221 and at 222, the gesture-touch screen managermaps the reference key of a model that includes a plurality of keys,each key having a respective key value and respective location fixedrelative to the reference key.

In an embodiment of 222 and at 223, the gesture-touch screen managerprovides the mode as a numeric keypad. The reference key is a 5 key andother values include 1-4 keys, 6-9 keys, 0 (zero) key, enter key, andcancel key. The keys are arranged in horizontal rows and verticalcolumns corresponding to an x-axis and a y-axis, respectively.

In an embodiment, of 223, at 224, the gesture-touch screen managerprovides the reference key such that the reference key is independent ofa position of initial contact of the touch on the touchscreen.

At 230, the gesture-touch screen manager tracks movement of the contacton the touchscreen during the touch.

At 240, the gesture-touch screen manager resolves the touch from 230into a gesture.

At 250, the gesture-touch screen manager assigns a key entry value forthe touch based on a relative location of the reference key and thegesture.

In an embodiment of 222 and 230, at 251, the gesture-touch screenmanager links at least one unique gesture to each key value. Eachgesture is defined by one or more changes in said movement along thex-axis or the y-axis.

In an embodiment of 251 and at 252, the gesture-touch screen managerassociated two changes detected along a same axis with a special keyentry value.

In an embodiment of 252 and at 253, the gesture-touch screen managerassigns the special key entry value to the key entry value as an enterkey. Processing ends for detection on the gesture-based touches(gestures) upon detection of an entered enter key. The enter key isrecognized when the two changes represent an initial movement in onehorizontal direction to the left along the x-axis followed by a last(second) movement in direction to an opposite horizontal direction alongthe x-axis.

In an embodiment of 252 and at 254, the gesture-touch screen managerassigns the special key entry value as a cancel key that erases previousdetected gesture-based touches made on the touchscreen. The cancel keyis recognized when the two successive single touches made within athreshold period of time (a double tap touch).

According to an embodiment, at 260, the gesture-touch screen manageriterates back to 210 until an enter key is detected within a givengesture-based touch, which provides an indication that a PIN has beencompletely entered by a user touching the touchscreen with thegesture-based touches.

In an embodiment of 260, and at 261, the gesture-touch screen managerencrypts the key entry value as an encrypted PIN, provides the encryptedPIN to a transaction processor of a transaction terminal, andrelinquishes control of the touchscreen to the transaction terminal forcontinued processing on a transaction at the transaction terminal withthe encrypted PIN.

FIG. 3 is a diagram of another method 300 for processing a gesturaltouch interface on a touch display, according to an example embodiment.The software module(s) that implements the method 300 is referred to asa “touch-based gesture recognizer.” The touch-based gesture recognizeris implemented as executable instructions programmed and residing withinmemory and/or a non-transitory computer-readable (processor-readable)storage medium and executed by one or more processors of a device. Theprocessors that execute the touch-based gesture recognizer arespecifically configured and programmed to process the touch-basedgesture recognizer. The touch-based gesture recognizer may have accessto one or more network connections during its processing. The networkconnections can be wired, wireless, or a combination of wired andwireless.

In an embodiment, the device that executes the touch-based gesturerecognizer is the transaction terminal 110. In an embodiment, thetransaction terminal 110 is one of: a SST, a POS terminal, an ATM,and/or a kiosk.

In an embodiment, the device that executes the touch-based gesturerecognizer is one of: a table, a phone, a laptop, a desktop, and awearable processing device.

In an embodiment, the touch-based gesture recognizer is all of somecombination of: gestural PIN translator 141 and/or the method 200.

In an embodiment, the touch-based gesture recognizer presents anotherand in some ways enhanced processing perspective from that which wasshown in the method 200 of the FIG. 2.

At 310, the touch-based gesture recognizer places a touchscreen in agesture-based mode of operation. This can be secure mode or non-securemode and processed by a non-secure processor or a secure processor asdiscussed above with the FIGS. 1A-1B. In an embodiment, the touch-basedgesture recognizer is places the touchscreen in the gesture-based modeof operation upon detection of a connected audio device made to thedevice/terminal that executes the touch-based gesture recognizer. Thiscan be a wired headphone connection or can be wireless connection wherethe speaker output is being wirelessly sent to a wirelessly connecteddevice.

In an embodiment, at 311, the touch-based gesture recognizer renders andmaintains a blank screen for visual presentation on the touchscreenduring that gesture-based mode of operation. That is, there is no visualdata presented on the touchscreen while the touchscreen is in thegesture-based mode of operation. Alternatively, visual data may bepresented but it is completely ignored while the touchscreen is in thegesture-based mode of operation.

At 320, the touch-based gesture recognizer identifies gesture-basedtouches made on a surface of the touchscreen, each gesture-based touchis a continuous touch that retains contact with the surface of thetouchscreen and is considered to be completed when contact is released(finger is lifted) from the surface.

At 330, the touch-based gesture recognizer maintains any directionalchanges made in movements for each gesture-based touch along an x-axisand a y-axis. Each gesture-based touch is assumed to begin at areference key location regardless of an initial position detected forthat gesture-based touch on the surface. That is, the user can placehis/her finger at any position on the surface of the touchscreen andinitial contact is assumed to be the reference key location. Thepositions touched by the finger on the touchscreen's surface can changedwith each separate gesture-based touch. Each gesture-based touch isassumed to being at the reference key location.

At 340, the touch-based gesture recognizer assigns each gesture-basedtouch to a specific key value based on the reference key location, thedirectional changes in movements, if any, and a key layout model.

In an embodiment, at 341, the touch-based gesture recognizer assignseach specific gesture-based touch to a reference key value when thespecific gesture-based touch lacks any directional change in movements.Here, the user touches the surface of the touchscreen and removes hisfinger with no continuous contact with the surface accompanied by anymovements along the x-axis or y-axis (there is also no detection of asecond touch within a short period of time, which would be recognized asa cancel key). The length of time that the user maintains contact withthe surface is irrelevant, such that a press and hold and then releaseis still associated with the reference key location; however, if thelength of time is very short and immediately followed by a second singletouch then the touch-based gesture recognizer identifies this as acancel key being entered.

In an embodiment, at 342, the touch-based gesture recognizer matcheseach set of direction changes in movements associated with eachgesture-based touch as a path for moving from the reference key locationwithin the layout model to an ending key location within the layoutmodel. The length of each drawn line along the x-axis and the y-axis isalso irrelevant; rather it is simply the movement detected along thex-axis and/or y-axis and the number of different movements on the twoaxis.

In an embodiment, at 343, the touch-based gesture recognizer erasesprevious specific key values assigned during the gesture-based mode ofoperation when the given special key value is assigned to a cancel key.

In an embodiment, at 344, the touch-based gesture recognizer providesconfirmatory audio tone-based feedback as each gesture-based touch isassigned a corresponding specific key value.

In an embodiment of 344 and at 345, the touch-based gesture recognizerprovides warning audio tone-based feedback when any gesture is unable tobe assigned to any of the specific key values. This allows the user totry again.

At 350, the touch-based gesture recognizer provides the specific keyvalues as a PIN when a given gesture-based touch is assigned to an enterkey.

At 360, the touch-based gesture recognizer causes the touchscreen toexit the gesture-based touch mode of operation after 350. That is, ifthe touch-based gesture recognizer is being processed on a devicethrough a secure processor that is separate from a transactionprocessor, the touch-based gesture recognizer relinquishes control ofthe touchscreen back to the transaction processor. A different interfacemay then be rendered on the screen or alternatively another instance ofthe touch-based gesture recognizer may initiated for continuedgesture-based processing on the device.

FIG. 4 is a diagram of a terminal 400 for processing a gestural touchinterface on a touch display, according to an example embodiment. Theterminal 400 includes a variety of hardware components and softwarecomponents. The software components of the terminal 400 are programmedand reside within memory and/or a non-transitory computer-readablemedium and execute on one or more processors of the terminal 400. Theterminal 400 may communicate over one or more networks, which can bewired, wireless, or a combination of wired and wireless.

In an embodiment, the terminal 400 implements, inter alia, theprocessing described above with the FIGS. 1A-1B and 2-3.

The terminal 400 is the transaction terminal 110.

In an embodiment, the terminal 400 is one of: an ATM, a SST, a POSterminal, a phone, a tablet, a laptop, a desktop computer, and awearable processing device.

The terminal 400 includes a touchscreen display 401 and a gesture touchmanager 402. The terminal 400 also includes at least one hardwareprocessor and non-transitory computer-readable storage medium/memory.The non-transitory computer-readable storage medium/memory includesexecutable instruction that represent the gesture touch manager 402. Theexecutable instructions executed by the at least one hardware processorof the terminal 400 from the non-transitory computer-readable storagemedium/memory.

In an embodiment, the touchscreen display is the touchscreen display120.

In an embodiment, the gesture-touch manager 402 is all or somecombination of the gestural PIN translator 141, the method 200, and/orthe method 300.

The gesture-touch manager 402, when executed by the hardware processorof the terminal 400 from the non-transitory computer-readable storagemedium, is configured to perform processing to: identify gesture-basedtouches made on a surface of the touchscreen 401 when a touch contactwith the surface is released regardless of initial starting positionsfor each gesture-based touch made on the surface; assign eachgesture-based touch to an input value based on a mapping that associatesthe starting positions with a single reference input value, anddirectional movements recorded for each gesture-based touch identifyinga path to the corresponding input value from the single reference inputvalue; and provide a set of assigned input values as PIN for atransaction being processed on the terminal 400.

It is to be noted that although the various examples presented werewithin the context of online middleware food service providers, otherembodiments of the invention are not so limited, such that any retailermiddleware service that sells products other than food can benefit fromthe teachings presented herein and above.

It should be appreciated that where software is described in aparticular form (such as a component or module) this is merely to aidunderstanding and is not intended to limit how software that implementsthose functions may be architected or structured. For example, modulesare illustrated as separate modules, but may be implemented ashomogenous code, as individual components, some, but not all of thesemodules may be combined, or the functions may be implemented in softwarestructured in any other convenient manner.

Furthermore, although the software modules are illustrated as executingon one piece of hardware, the software may be distributed over multipleprocessors or in any other convenient manner.

The above description is illustrative, and not restrictive. Many otherembodiments will be apparent to those of skill in the art upon reviewingthe above description. The scope of embodiments should therefore bedetermined with reference to the appended claims, along with the fullscope of equivalents to which such claims are entitled.

In the foregoing description of the embodiments, various features aregrouped together in a single embodiment for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting that the claimed embodiments have more features than areexpressly recited in each claim. Rather, as the following claimsreflect, inventive subject matter lies in less than all features of asingle disclosed embodiment. Thus the following claims are herebyincorporated into the Description of the Embodiments, with each claimstanding on its own as a separate exemplary embodiment.

The invention claimed is:
 1. A method, comprising: detecting a touch ofa single finger on a touchscreen of a device, the touch comprising acontact with the touchscreen followed by a release of said contact fromthe touchscreen, wherein detecting the touch further includes detectingthe touch even when the touchscreen is presenting visual information;associating the contact with a reference key; tracking a movement of thecontact on the touchscreen during said touch, wherein a length of timeof the touch before the release is irrelevant to the tracking of themovement of the touch relative to the reference key in order to resolvethe touch into a user-provided key value, and wherein a magnitude of thetouch in any direction is irrelevant to the tracking of the movement ofthe touch relative to the reference key in order to resolve the touchinto the user-provided key value; resolving the tracking into a gesturethat represents the user-provided key value; assigning a key entry valuefor the touch based on a relative location of the reference key and thegesture, wherein when the touch comprised no tracked movement with nomagnitude in any direction, the key entry value is assigned to areference key value associated with the reference key, wherein the keyentry value represents the user-provided key value; iterating back tothe detecting until an enter key is detected, wherein iterating furtherincludes encrypting the key entry value as an encrypted PIN, providingthe encrypted PIN to a transaction processor of a transaction terminal,and relinquishing processing control of the touchscreen to thetransaction terminal for continued processing on a transaction at thetransaction terminal with the encrypted PIN.
 2. The method of claim 1,wherein associating further includes ignoring subsequent touches on thetouchscreen until an audio device connected to said device is detected.3. The method of claim 1, wherein associating further comprises mappingthe reference key to one key of a model that includes a plurality ofkeys each having a respective key value and a respective location fixedrelative to the reference key.
 4. The method of claim 3, wherein themodel is configured as a numeric keypad wherein the reference key is a 5key and the other values include 1-4 keys, 6-9 keys, 0 (zero) key, enterkey, and cancel key, the keys arranged in horizontal rows and verticalcolumns corresponding to an x-axis and a y-axis, respectively.
 5. Themethod of claim 4, wherein mapping the reference key is independent of aposition of said contact of the touch on the touchscreen.
 6. The methodof claim 4, wherein assigning further includes linking at least oneunique gesture to each key entry value, wherein each gesture is definedby one or more changes in said movement along the x-axis or y-axis. 7.The method of claim 6, wherein assigning further includes associatingtwo changes detected along a same axis with a special key entry value.8. The method of claim 7, wherein assigning further includes assigningthe special key entry value to the key entry value as an enter key thatends gesture-based touches on the touchscreen when the two changesrepresent an initial movement made in a first horizontal direction alongthe x-axis that is followed by a second movement made in an oppositehorizontal direction along the x-axis.
 9. The method of claim 7, whereinassigning further includes assigning the special key entry value to thekey entry value as a cancel key that erases previous gesture-basedtouches detected on the touchscreen when the two successive touches aredetected with no directional changes within a threshold period of timeas a double tapped touch.